Video coding device and video decoding device

ABSTRACT

A video coding device capable of adaptively processing input video data according to property of the data and realizing effective progressive transmission of the coded data even if image components are different in size and/or different in the number of subbands. The above-mentioned object can be realized by the provision of a transferring-order deciding and ranging portion that can prepare an integrated component unit by forming combinations of subband-based frequency-coefficients of respective components Y, U and V and can change the number of respective elements of respective components Y, U and V.

TECHNICAL FIELD

The present invention pertains to the field of digital video processingand relates to a video coding device for efficiently encoding video dataand a video decoding device for decoding video data coded by the videocoding device.

BACKGROUND ART

Recently, there has been proposed a subband coding method that canefficiently encode and decode video signals. The well-knownhigh-efficient subband encoding method is used to decompose an inputimage into frequency bands by a bank of band-decomposing filters. Theband-decomposing filter-bank is a one-dimensional filter-bank that canserve as a two-dimensional band-decomposing filter-bank by repeatingprocessing the input image in horizontal and vertical directions. Thismethod was reported by Fujii, Noumura. “Topics on Wavelet Transform” ina Report of “TECHNICAL REPORT of IEICE, IE92-11, 1992”.

In the prior art, a subband image as shown in FIG. 1B obtained byconducting two-dimensional subband decomposition three times. The firsttwo-dimensional subband decomposition obtains a horizontal high-pass anda vertical low-pass band, a horizontal low-pass and vertical high-passband and a horizontal and vertical high-pass band, which are designatedby HL1, LH1 and HH1 respectively. A horizontal and vertical low-passband obtained by the first decomposition is further subjected totwo-dimensional band-decomposition by which three subbands HL2, LH2 andHH2 are obtained.

A horizontal and vertical low-pass subband obtained by the seconddecomposition is further subjected to third two-dimensional subbanddecomposition by which three subbands HL3, LH3 and HH3 and a horizontaland vertical low-pass subband LL3 are obtained. A Wavelet-convertingfilter-bank or a band-decomposing and synthesizing filter-bank may beused as the band-decomposing filter-bank. Thus, the decomposedsubband-images are of a hierarchical (layer) structure fromlow-frequency band to high-frequency band.

Progressive image transmitting can be easily realized utilizing thehierarchical structure of the subband images. The progressive imagetransmitting method enables a video decoding device to reproduce alow-resolution image by using only a part of coded data. The more codeddata is reproduced, the higher resolution the decoded image has.Japanese Laid-Open Patent Publication (TOKKAI HEI) No. 8-242379describes a system (referred hereinafter to as a prior art system) torealize the progressive image transmitting.

A video coding device using in the prior art system comprises a subbanddecomposing portion for decomposing an input image into subband imagesby using two-dimensional decomposing filters, a coefficient codingportion for encoding coefficients of the decomposed subband images, avariable-length coding portion for performing variable-length coding ofthe coded coefficient data from the coefficient coding portion and aline-transmitting portion for transmitting a plurality of componentscomposing the image per line at a time. The coefficient coding portionperforms encoding the coefficients by using any one of various kinds ofcoding methods (e.g., DPCM coding, zero-tree coding, andscalar-quantizing coding). This process includes a quantizing step.

The operation of the line transmitting portion will be described belowin detail, by way of example, with an input image composed ofthree-components Y (a luminance component) and U, V (chrominancecomponents) and being conducted subband decomposition three times asshown in FIG. 1B. Processing starts from a subband LL3, which gives thelowest resolution of the image.

In the example, the line-transmitting portion transmits the componentsY, U and V sequentially line by line in the order from the first line ofthe subband LL3. Having transferred all lines of the subband LL3, theportion transfers the components Y, U and V in the subbands LH3, HL3 andHH3 respectively in the order: the component Y on the first lines of thesubbands LH3, HL3 and HH3; the component U on the first lines of thesubbands LH3, HL3 and HH3; the component V on the first lines of thesubbands LH3, HL3 and HH3; the component Y on the second lines of thesubbands LH3, HL3 and HH3; U on the second lines of LH3, HL3 and HH3; Von the second lines of the subbands LH3, HL3 and HH3 and so on. Havingtransmitted all lines of LH3, HL3 and HH3, the line transmitting portiontransfers, in similar way, lines of LH2, HL2; HH2 and, then, lines ofLH1, HL1, HH1. The above-mentioned procedure of the line-transmittingportion is executed according to a programed flow.

Orderly transmission of the components Y, U, V composing the image perline produces coded data having a hierarchical structure.

The prior art video decoding device comprises a line receiving portionfor receiving the coded data from the line-transmitting portion of thevideo-coding device above-mentioned and rearranging the data torespective component groups, a variable-length decoding portion fordecoding the rearranged variable-length-coded data, a decoded datacounting portion for counting bits of data decoded by thevariable-length decoding portion, a decoding truncating portion forcomparing the number of the bits counted by the decoded-data countingportion with a preset threshold or an externally-given threshold to givea command for stopping the decoding operation of the variable-lengthdecoding portion when the number of decoded bits exceeds the threshold,a data completing portion for compensating for lack of truncated data byadding zero when having truncated the decoding the coded data at thespecified number of bits, a coefficient decoding portion for decodingcoded coefficient data by reversing the same processing procedure of thecoefficient coding portion and a subband synthesizing portion forsynthesizing an image from the subbands through two-dimensionalsynthesizing filters.

The video decoding device can thus reproduce an entire image from codeddata having a hierarchical structure or a part thereof.

The conventional video-coding and video-decoding system can realizeprogressive image transmitting by transmitting image components per linein an ascending order starting from the lowest-resolution band-image.However, the prior art system encounters several inconvenient problemsresulting from the fixed transfer-unit of a line. For example, an imagecomposed of luminance component Y and chrominance components U and V maybe easier recognized by transmitting only the component Y before thecomponents U and V rather than transmitting all components as a unit.

In this case, it is preferable to transfer the image components subbandby subband, not by line. Furthermore, it is proved that an imagecomposed of components R, G, B may be reproduced with better subjectiveimage-quality at the decoding terminal when coded coefficients of therespective components R, G and B are transmitted one by one. This isbecause these components have substantially the same influence on thevisual property.

The prior art system presumes that components of an image have the samesize. Therefore, it cannot be adaptable to an input image composed ofdifferent sizes of components in format of, e.g., 4:2:2 or 4:2:0.

Furthermore, the prior art system presumes that respective components ofan image have the same number of subbands and cannot be adaptable to aninput image whose components are divided into different numbers ofsubbands.

DISCLOSURE OF INVENTION

The present invention is directed to a system for effective progressiveimage transmitting by solving the foregoing problems involved in theprior arts.

(1) Accordingly, an object of the present invention is to provide avideo coding device, which is provided with a subband-decomposing meansfor decomposing an image being composed of N (N≧2) kinds of luminance orchrominance components into subband images for each of components A^(n)(1≦n≦N, where n is an integer) composing an image to be coded,coefficient coding means for encoding a frequency coefficient of thesubband images, rearranging means for preparing integrated componentunits by combining frequency coefficients included in respectivecomponents A^(n) according to the coded coefficient data prepared by thecoefficient coding means and rearranging the prepared integratedcomponent units of the coefficient-coded data in an ascending order ofsubband image resolution, starting from the integrated component unitincluding the coded coefficient data of the lowest resolution subband,and a variable-length coding means for performing variable-lengthencoding of the rearranged coefficient-coded data, wherein therearranging means prepares each of the integrated component units bysetting therein the frequency coefficients contained in the respectivecomponents A^(n), which are all frequency-coefficients included in m(m≧1) pieces of the respective components' subbands, when the componentsA^(n) are have the same size and the same number of subbands.

(2) Correspondingly, another object of the present invention is toprovide a video decoding device, which is provided with avariable-length decoding means for decoding variable-length coded data,a decoded-data counting means for counting bits of each integratedcomponent unit decoded by the variable-length decoding means, a decodingtruncating means for comparing the number of bits counted by thedecoded-data counting means with an externally-given number of bits andgiving a decoding-stop command when the number of decoded bits exceedsthe given number of bits, a component separating means for separatingthe decoded integrated component unit into respective components A^(n),a data completing means for compensating for lack of truncated data byadding a specified value to each of the components composing a screenfulimage, data arranging means for arranging coded coefficient dataseparated by the component separating means into specified positions forrespective components A^(n), a coefficient decoding means for decodingcoded-coefficient data separated and arranged for respective componentsA^(n) by the component separating means, and a subband synthesizingmeans for reproducing a decoded image by combining subbands of datadecoded by the coefficient decoding means for respective componentsA^(n), wherein the component separating means separates the integratedcomponent unit as combinations of all frequency coefficients containedin m (m≧1) subbands for respective components A^(n) when the respectivecomponents A^(n) have the same size and the same number of subbands.

The integrated component units contains all frequency coefficients in m(m≧1) respective subbands of respective component A^(n). Therefore,specified subbands of the image components such as luminance signal Yand chrominance signals U and V that may have different levels ofinfluence on human visual property can be transmitted first to enableone to recognize a summary of the image at an earlier stage of decodingat the decoding side. When a codable image is known to be of higherresolution in a specified direction, the coding device can transmitfirst coded coefficients of higher-resolution-direction subbands and thedecoding device can decode those coded coefficients, terminate thedecoding in the midway of decoding all coded data and reproduce theimage from only data decoded till that time to improve subjective-imagequality of the image.

(3) Another object of the present invention is to provide a video codingdevice, which is provided with a subband-decomposing means fordecomposing an image being composed of N (N≧2) kinds of luminance orchrominance components into subband images for each of components A^(n)(1≦n≦N, where n is an integer) composing an image to be coded,coefficient coding means for encoding a frequency coefficient of thesubband images, rearranging means for preparing integrated componentunits by combining frequency coefficients included in respectivecomponents A^(n) according to the coded coefficient data prepared by thecoefficient coding means and rearranging the prepared integratedcomponent units of the coefficient-coded data in an ascending order ofsubband image resolution, starting from the integrated component unitincluding the coded coefficient data of the lowest resolution subband,and a variable-length coding means for performing variable-lengthencoding of the rearranged coefficient-coded data, wherein therearranging means prepares each of the integrated component units bysetting therein the frequency coefficients included in the respectivecomponents A^(n) as m (m≧1) pieces of frequency-coefficients containedat the same relative positions in m (m≧1) pieces of the respectivecomponents' subbands when the components A^(n) have the same size andthe same number of subbands.

(4) Correspondingly, another object of the present invention is toprovide a video decoding device, which is provided with avariable-length decoding means for decoding variable-length coded data,a decoded-data counting means for counting bits of each integratedcomponent unit decoded by the variable-length decoding means, a decodingtruncating means for comparing the number of bits counted by thedecoded-data counting means with an externally-given number of bits andgiving a decoding-stop command when the number of decoded bits exceedsthe given number of bits, a component separating means for separatingthe decoded integrated component unit into respective components A^(n),a data completing means for compensating for lack of truncated data byadding a specified value to each of the components composing a screenfulimage, data arranging means for arranging coded coefficient dataseparated by the component separating means into specified positions forrespective components A^(n), a coefficient decoding means for decodingcoded-coefficient data separated and arranged for respective componentsA^(n) by the component separating means, and a subband synthesizingmeans for reproducing a decoded image by combining subbands of datadecoded by the coefficient decoding means for respective componentsA^(n), wherein the component separating means separates the integratedcomponent unit into combinations of m (m≧1) pieces of frequencycoefficients having the same relative positions in respective m (m≧1)subbands of the respective components A^(n) when the components A^(n)have the same size and the same number of subbands.

Therefore, the devices operate with integrated component units whoseelements are m (m≧1) pieces of frequency coefficients having the samerelative positions in m (m≧1) respective subbands of respectivecomponents A^(n) and can decode those coded coefficients, terminate thedecoding in the midway of decoding all the coded data and reproduce theimage from only the data decoded till that time to improvesubjective-image quality of the image when the image is composed ofcomponents. R, G and B that have substantially almost the same influenceon human visual property.

(5) Another object of the present invention is to provide a video codingdevice, which is provided with a subband-decomposing means fordecomposing an image being composed of N (N≧2) kinds of luminance orchrominance components into subband images for each of components A^(n)(1≦n≦N, where n is an integer) composing an image to be coded,coefficient coding means for encoding a frequency coefficient of thesubband images, rearranging means for preparing integrated componentunits by combining frequency coefficients included in respectivecomponents A^(n) according to the coded coefficient data prepared by thecoefficient coding means and rearranging the prepared integratedcomponent units of the coefficient-coded data in an ascending order ofsubband image resolution, starting from the integrated component unitincluding the coded coefficient data of the lowest resolution subband,and a variable-length coding means for performing variable-lengthencoding of the rearranged coefficient-coded data, wherein therearranging means prepares each of the integrated component units bysetting therein the different number of frequency-coefficients in therespective components A^(n) according to each component size when thecomponents A^(n) are different in size and have the same number ofsubbands.

(6) Correspondingly, another object of the present invention is toprovide a video decoding device, which is provided with avariable-length decoding means for decoding variable-length coded data,a decoded-data counting means for counting bits of each integratedcomponent unit decoded by the variable-length decoding means, a decodingtruncating means for comparing the number of bits counted by thedecoded-data counting means with an externally-given number of bits andgiving a decoding-stop command when the number of decoded bits exceedsthe given number of bits, a component separating means for separatingthe decoded integrated component unit into respective components A^(n),a data completing means for compensating for lack of truncated data byadding a specified value to each of the components composing a screenfulimage, data arranging means for arranging coded coefficient dataseparated by the component separating means into specified positions forrespective components A^(n), a coefficient decoding means for decodingcoded-coefficient data separated and arranged for respective componentsA^(n) by the component separating means, and a subband synthesizingmeans for reproducing a decoded image by combining subbands of datadecoded by the coefficient decoding means for respective componentsA^(n), wherein the component separating means separates the integratedcomponent unit as combinations of different pieces of frequencycoefficients according to respective component sizes when the respectivecomponents A^(n) are different in size and have the same number ofsubbands.

The devices can be adapted to process an image whose luminance andchrominance components are different from each other by resolution,having the great advantage over the conventional method that can beapplied to an image whose components have the same resolution. Thisfeature provided by the present invention is desirable in particular todigital image processing since many digital images are usually formattedto have higher resolution of the luminance component than that ofchrominance component.

(7) Another object of the present invention is to provide a video codingdevice, which is provided with a subband-decomposing means fordecomposing an image being composed of N (N≧2) kinds of luminance orchrominance components into subband images for each of components A^(n)(1≦n≦N, where n is an integer) composing an image to be coded,coefficient coding means for encoding a frequency coefficient of thesubband images, rearranging means for preparing integrated componentunits by combining the subbands included in respective components A^(n)according to the coded coefficient data prepared by the coefficientcoding means and rearranging the prepared integrated component units ofthe coefficient-coded data in an ascending order of subband imageresolution, starting from the integrated component unit including thecoded coefficient data of the lowest resolution subband, and avariable-length coding means for performing variable-length encoding ofthe rearranged coefficient-coded data, wherein the rearranging meansprepares each of the integrated component units by combining the samenumber of low-resolution subbands and the different number ofhigh-resolution subbands of the respective components A^(n) when thecomponents A^(n) are different in size and different in the number ofsubbands.

(8) Correspondingly, another object of the present invention is toprovide a video decoding device, which is provided with avariable-length decoding means for decoding variable-length coded data,a decoded-data counting means for counting bits of each integratedcomponent unit decoded by the variable-length decoding means, a decodingtruncating means for comparing the number of bits counted by thedecoded-data counting means with an externally-given number of bits andgiving a decoding-stop command when the number of decoded bits exceedsthe given number of bits, a component separating means for separatingthe decoded integrated component unit into respective components A^(n),a data completing means for compensating for lack of truncated data byadding a specified value to each of the components composing a screenfulimage, data arranging means for arranging coded coefficient dataseparated by the component separating means into specified positions forrespective components A^(n), a coefficient decoding means for decodingcoded-coefficient data separated and arranged for respective componentsA^(n) by the component separating means, and a subband synthesizingmeans for reproducing a decoded image by combining subbands of datadecoded by the coefficient decoding means for respective componentsA^(n), wherein the component separating means separates the integratedcomponent unit as combinations of the same number of low-resolutionsubbands and the different number of high-resolution subbands ofrespective components A^(n) when the respective components A^(n) aredifferent in size and different in the number of subbands.

The devices can be adapted to process an image whose luminance andchrominance components are different from each other by resolution andhave different subband-decomposition levels, getting a great advantageover the conventional method that can be applied to an image whosecomponents have the same resolution and the same number of subbands.This feature provided by the present invention is desirable inparticular to digital image processing since many digital images areusually formatted to have higher resolution of the luminance componentthan that of the chrominance component and it is general to vary thesubband-decomposition level according to the resolution of thecomponent.

(9) Another object of the present invention is to provide a video codingdevice, which is provided with a subband-decomposing means fordecomposing an image being composed of N (N≧2) kinds of luminance orchrominance components into subband images for each of components A^(n)(1≦n≦N, where n is an integer) composing an image to be coded,coefficient coding means for encoding a frequency coefficient of thesubband images, rearranging means for preparing integrated componentunits by combining the subbands included in respective components A^(n)according to the coded coefficient data prepared by the coefficientcoding means and rearranging the prepared integrated component units ofthe coefficient-coded data in an ascending order of subband imageresolution, starting from the integrated component unit including thecoded coefficient data of the lowest resolution subband, and avariable-length coding means for performing variable-length encoding ofthe rearranged coefficient-coded data, wherein the rearranging meansprepares each of the integrated component units by combining the samenumber of high-frequency subbands and the different number oflow-frequency subbands of the respective components A^(n) when thecomponents A^(n) are different in size and different in the number ofsubbands.

(10) Correspondingly, another object of the present invention is toprovide a video decoding device, which is provided with avariable-length decoding means for decoding variable-length coded data,a decoded-data counting means for counting bits of each integratedcomponent unit decoded by the variable-length decoding means, a decodingtruncating means for comparing the number of bits counted by thedecoded-data counting means with an externally-given number of bits andgiving a decoding-stop command when the number of decoded bits exceedsthe given number of bits, a component separating means for separatingthe decoded integrated component unit into respective components A^(n),a data completing means for compensating for lack of truncated data byadding a specified value to each of the components composing a screenfulimage, data arranging means for arranging coded coefficient dataseparated by the component separating means into specified positions forrespective components A^(n), a coefficient decoding means for decodingcoded-coefficient data separated and arranged for respective componentsA^(n) by the component separating means, and a subband synthesizingmeans for reproducing a decoded image by combining subbands of datadecoded by the coefficient decoding means for respective componentsA^(n), wherein the component separating means separates the integratedcomponent unit as combinations of the same number of high-resolutionsubbands and the different number of low-resolution subbands ofrespective components A^(n) when the components A^(n) are differentsizes and different in the number of subbands.

The devices can be adapted to process an image whose luminance andchrominance components have different resolution levels and differentsubband-decomposition levels, getting a great advantage over theconventional method that can be applied to an image whose componentshave the same resolution and the same number of subbands. Furthermore,these aspects of the present invention provide such a feature that eachintegrated component unit always reflects the ratio of numbers ofrespective components contained in an input image. This featureeliminates the need for decoding redundant data at the decoding sidewhen decoding the amount of data according to the resolution of thedisplay unit.

(11) Another object of the present invention is to provide a videocoding device, which is based on the device of (9) above-mentioned andfurther characterized in that the rearranging means prepares each of theintegrated component units by combining lowest ones of resolutionsubbands of the respective components A^(n) and different numbers of allother low-resolution-level subbands of the respective components A^(n)when the respective components A^(n) are different in size and differentin the number of subbands.

(12) Correspondingly, the present invention also provides avideo-decoding device, which is based on the device of (10)above-mentioned and further characterized in that the componentseparating means separates the integrated component unit intocombinations of subbands for respective components, each combinationcomposed of one lowest resolution subband and the different numbers ofall other low-resolution subbands.

The devices can first separate and transmit lowest-resolution subbandsof respective components A^(n) to first give a summary content of animage, making it possible to improve subjective quality of thereproduced image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a view for explaining a subband coding method.

FIG. 1B is a view for explaining a subband coding method.

FIG. 2 is a view for explaining a progressive video transmission system.

FIG. 3 is a block-diagram of a prior art video coding device.

FIG. 4 is a block-diagram of a prior art video decoding device.

FIG. 5 depicts a sequence of transferring subband image coefficientsaccording to the prior art video coding device.

FIG. 6 is a flow chart for explaining the operation of a prior art videocoding device.

FIG. 7 is a block-diagram of a video coding device which is a firstembodiment of the present invention.

FIG. 8 is a block-diagram of a video decoding device which is a firstembodiment of the present invention.

FIG. 9 depicts an example of decomposing image components into subbandsin the first embodiment of the present invention.

FIG. 10 depicts an example of a sequence of transferring coefficients ofsubband images in the first embodiment of the present invention.

FIG. 11 depicts another example of a sequence of transferringcoefficients of subband images in the first embodiment of the presentinvention.

FIG. 12 depicts an exemplified sequence of scanning coefficients inrespective subbands in the first embodiment of the present invention.

FIG. 13 depicts an example of collecting a plurality of subbands intoelements of an integrated component unit in the first embodiment of thepresent invention.

FIG. 14 depicts an example of using coefficients of each line in eachsubband as an element of an integrated component unit in the firstembodiment of the present invention.

FIG. 15 is a flow chart depicting a procedure of operations of a videocoding device which is the first embodiment of the present invention.

FIG. 16 is a flow chart depicting a procedure of operations of a videocoding device which is the first embodiment of the present invention.

FIG. 17 depicts an example of decomposing image components into subbandsin the second embodiment of the present invention.

FIG. 18 depicts a relationship between coefficients of respectivecomponents in the second embodiment of the present invention.

FIG. 19 depicts an example of an integrated component unit used in thesecond embodiment of the present invention.

FIG. 20 depicts another example of an integrated component unit used inthe second embodiment of the present invention.

FIG. 21 depicts an example of subband decomposition and an integratedcomponent unit used in the third embodiment of the present invention.

FIG. 22 depicts another example of subband decomposition and anintegrated component unit used in the third embodiment of the presentinvention.

BEST MODE FOR CARRYING OUT THE INVENTION

Prior to explaining preferred embodiments of the present invention,prior art video coding device and video decoding device will bedescribed below as references for the present invention.

Recently, there has been proposed a subband coding method that canefficiently encode and decode video signals. The well-knownhigh-efficient subband encoding method is used to decompose an inputimage into such frequency bands as shown in FIG. 1B by a bank ofband-decomposing filters as shown in FIG. 1A. The band-decomposingfilter-bank shown in FIG. 1A is a one-dimensional filter-bank that canserve as a two-dimensional band-decomposing filter-bank by repeatingprocessing the input image in horizontal and vertical directions. Thismethod was reported by Fujii, Noumura. “Topics on Wavelet Transform” ina Report of “TECHNICAL REPORT of IEICE, IE92-11, 1992”.

In FIG. 1A, there is shown a subband image obtained by conductingtwo-dimensional subband decomposition three times. The firsttwo-dimensional subband decomposition obtains a horizontal high-pass anda vertical low-pass band, a horizontal low-pass and vertical high-passband and a horizontal and vertical high-pass band, which are designatedby HL1, LH1 and HH1 respectively. A horizontal and vertical low-passband obtained by the first decomposition is further subjected totwo-dimensional band-decomposition by which three subbands HL2, LH2 andHH2 are obtained.

A horizontal and vertical low-pass subband obtained by the seconddecomposition is further subjected to third two-dimensional subbanddecomposition by which three subbands HL3, LH3 and HH3 and a horizontaland vertical low-pass subband LL3 are obtained. A Wavelet-convertingfilter-bank or a band-decomposing and synthesizing filter-bank may beused as the band-decomposing filter-bank. Thus, the decomposedsubband-images are of a hierarchical (layer) structure fromlow-frequency band to high-frequency band.

Progressive image transmitting can be easily realized utilizing thehierarchical structure of the subband images. As shown in FIG. 2, theprogressive image transmitting method enables a video decoding device toreproduce a low-resolution image by using only a part of coded data. Themore coded data is reproduced, the higher resolution the decoded imagehas. Japanese Laid-Open Patent Publication (TOKKAI HEI) No. 8-242379describes a system (referred hereinafter to as a prior art system) torealize the progressive image transmitting, which structure is shown inFIGS. 3 and 4.

FIG. 3 shows a video coding device using in the prior art system andFIG. 4 shows a video decoding device using the system. The video codingdevice as shown in FIG. 3 comprises a subband decomposing portion 2001for decomposing an input image into subband images by usingtwo-dimensional decomposing filters, a coefficient coding portion 2002for encoding coefficients of the decomposed subband images, avariable-length coding portion 2003 for performing variable-lengthcoding of the coded coefficient data from the coefficient coding portion2002 and a line-transmitting portion 2004 for transmitting a pluralityof components composing the image per line at a time. The coefficientcoding portion 2002 performs encoding the coefficients by using any oneof various kinds of coding methods (e.g., DPCM coding, zero-tree coding,and scalar-quantizing coding). This process includes a quantizing step.

The operation of the line transmitting portion 2004 will be describedbelow in detail, by way of example, with an input image composed ofthree-components Y (a luminance component) and U, V (chrominancecomponents) and being conducted subband decomposition three times asshown in FIG. 1B. Processing starts from a subband LL3 shown in FIG. 1B,which gives the lowest resolution of the image.

As shown in FIG. 5, the line-transmitting portion 2004 transmits thecomponents Y, U and V sequentially line by line in the order from thefirst line of the subband LL3. Having transferred all lines of thesubband LL3, the portion transfers the components Y, U and V in thesubbands LH3, HL3 and HH3 respectively in the order: the component Y onthe first lines of the subbands LH3, HL3 and HH3; the component U on thefirst lines of the subbands LH3, HL3 and HH3; the component V on thefirst lines of the subbands LH3, HL3 and HH3; the component Y on thesecond lines of the subbands LH3, HL3 and HH3; U on the second lines ofLH3, HL3 and HH3; V on the second lines of the subbands LH3, HL3 and HH3and so on. Having transmitted all lines of LH3, HL3 and HH3, the linetransmitting portion transfers, in similar way, lines of LH2, HL2, HH2and, then, lines of LH1, HL1, HH1. The above-mentioned procedure of theline-transmitting portion 2004 is illustrated by a flowchart of FIG. 6.

Orderly transmission of the components Y, U, V composing the image perline produces coded data having a hierarchical structure.

Referring to FIG. 4, the video decoding device comprises a linereceiving portion 2004 for receiving the coded data from theline-transmitting portion 2004 of the video-coding device of FIG. 3 andrearranging the data to respective component groups, a variable-lengthdecoding portion 2101 for decoding the rearranged variable-length-codeddata, a decoded data counting portion 2102 for counting bits of datadecoded by the variable-length decoding portion, a decoding truncatingportion 2103 for comparing the number of the bits counted by thedecoded-data counting portion with a preset threshold or anexternally-given threshold to give a command for stopping the decodingoperation of the variable-length decoding portion 2101 when the numberof decoded bits exceeds the threshold, a data completing portion 2105for compensating for lack of truncated data by adding zero when havingtruncated the decoding the coded data at the specified number of bits, acoefficient decoding portion 2106 for decoding coded coefficient data byreversing the same processing procedure of the coefficient codingportion 2002 of FIG. 3 and a subband synthesizing portion 2107 forsynthesizing an image from the subbands through two-dimensionalsynthesizing filters.

The video decoding device can thus reproduce an entire image from codeddata having a hierarchical structure or a part thereof.

The conventional video-coding and video-decoding system can realizeprogressive image transmitting by transmitting image components per linein an ascending order starting from the lowest-resolution band-image.However, the prior art system encounters several inconvenient problemsresulting from the fixed transfer-unit of a line. For example, an imagecomposed of luminance component Y and chrominance components U and V maybe easier recognized by transmitting only the component Y before thecomponents U and V rather than transmitting all components as a unit.

In this case, it is preferable to transfer the image components subbandby subband, not by line. Furthermore, it is proved that an imagecomposed of components R, G, B may be reproduced with better subjectiveimage-quality at the decoding terminal when coded coefficients of therespective components R, G and B are transmitted one by one. This isbecause these components have substantially almost the same influence onthe visual property.

The prior art system presumes that components of an image have the samesize. Therefore, it cannot be adaptable to an input image composed ofdifferent sizes of components in format of, e.g., 4:2:2 or 4:2:0.

Furthermore, the prior art system presumes that respective components ofan image have the same number of subbands and cannot be adaptable to aninput image whose components are decomposed into different numbers ofsubbands.

Referring now to accompanying drawings, a video-coding device and avideo decoding device according to the present invention will bedescribed below in detail.

FIG. 7 is a block diagram showing a video coding device which is a firstembodiment of the present invention. As shown in FIG. 7, the firstembodiment of the present invention includes a subband decomposingportion (subband decomposing means) 101, a coefficient coding portion(coefficient coding means) 102 and a variable-length coding portion(variable-length coding means) 103, which are similar in construction toportions 2001, 2002 and 2003, respectively, of FIG. 3.

In FIG. 7, numeral 104 designates a transfer-order deciding andrearranging portion (rearranging means) that decides an integratedcomponent unit prepared by combining luminance or chrominance elementsfrom coded coefficient data provided from the coefficient coding portion102 and arranges coded coefficient data of subbands in a transmittingorder starting from the coded coefficient data of the lowest-resolutionsubband. Whereas the prior art video-coding device rearranges codedcoefficient data in the transmitting order after variable-length codingof the data, the video-coding device according to the present inventionrearranges the coded coefficient data in the transmitting order beforevariable-length coding of the data.

This makes it possible to conduct variable-length coding of the codedcoefficient data by, e.g., an arithmetic coding method besides theHuffman coding method. According to the present invention, it is alsopossible to conduct rearrangement of the coded coefficient data aftervariable-length coding as the prior art device does. The operation ofthe first embodiment is described below with an input image composed ofthree components Y (luminance), U (chrominance) and V (chrominance),which is the same as that used in the prior art device. In thisembodiment, these components have the same resolution, i.e., the sameimage sizes.

An integrated component unit may be prepared from coefficient-coded databy combining elements Y, U and V. The following example is an integratedcomponent unit that is prepared of subbands of Y, U and V.

FIG. 9 shows coefficients of subband images obtained through performingthree times of subband-decomposition of respective image components Y, Uand V ((a), (b) and (c) part of FIG. 9, respectively). The image sizesof the components Y, U and V are equal to each other. FIGS. 10 and 11show the order of transmitting the subband image coefficients of Y, Uand V ((a), (b) and (c) part of both of FIG. 10 and FIG. 11) in FIG. 9,respectively. Characters Y, U and V with numeral suffixes in both ofFIGS. 10, 11 denote the order of transmitting subbands of respectivecomponents.

In the first embodiment, an integrated component unit is composed ofsubbands and contains a set of the same-resolution subbands ofrespective components, that is: (Y_(i), U_(i), V_(i)) where i=1 to 10(the order of subbands to be transferred).

The transmitting order is as follows: (Y₁, U₁, V₁), (Y₂, U₂, V₂), . . ., (Y₁₀, U₁₀, V₁₀).

In the case of FIG. 10, a coefficient of high resolution in a horizontaldirection is transferred before a coefficient of high resolution invertical direction, while in the case of FIG. 11, a coefficient of highresolution in a vertical direction is transferred before a coefficientof high resolution in horizontal direction. Accordingly, the resolutionof the image reproduced at the decoding side can be improved first inthe horizontal direction in the case of FIG. 10 and first in thevertical direction in the case of FIG. 11.

Processing of coefficients within a subband may be performed in any ofthe orders shown in FIG. 12.

The subband is horizontally scanned from above left to below right (partdesignated by (1) of FIG. 12) or vertically scanned from above left tobelow right (part (2) of FIG. 12) or scanned spirally from the center ofthe subband to the outside thereof (part (3) of FIG. 12). In FIG. 12,coefficients in a subband are processed by one scanning as shown in eachpart (a) of parts (1) to (3) and coefficients in a subband are processedby scanning twice as shown in each parts (b) and (c) of the parts (1) to(3) respectively.

In part (c) in FIG. 12, each arrow shows a coefficient or a set ofplural (t) coefficients. A set of (t) coefficients is transferred in thedirection indicated by the arrow, a subsequent set of (t) coefficientsis not transferred and a further subsequent set of (t) coefficients istransferred in the direction indicated by the arrow. This steps arerepeated in first scanning process. The coefficients leftnot-transferred in the first scanning process (shown as arrows with reallines) are transferred in the second scanning process (shown as arrowswith broken lines) in the similar way as in the first scanning process.

As shown in parts (4) and (5) of FIG. 12, it is also possible to processcoefficients of a subband by scanning three or more times. For example,a horizontal interval and a vertical interval between coefficients to beprocessed by n-th scan are expressed as dx and dy respectively. Ifdx=dy=1, all coefficients are encoded by one raster scan. If dx=dy=2 andhorizontal and vertical positions of codable coefficients are determinedas (y, x), the first scan encodes coefficients at the positions (0, 0),(0, 2), (0, 4) . . . (2, 0), (2, 2), (2, 4) . . . (4, 0), (4, 2), (4, 4). . . ;

the second scan encodes coefficients at the positions (0, 1), (0, 3),(0, 5) . . . (2, 1), (2, 3), (2, 5) . . . (4, 1), (4, 3), (4, 5) . . . ;

the third scan encodes coefficients at the positions (1, 0), (1, 2), (1,4) . . . (3, 0), (3, 2), (3, 4) . . . (5, 0), (5, 2), (5, 4) . . . ;

the fourth scan encodes coefficients at the positions (1, 1), (1, 3),(1, 5) . . . (3, 1), (3, 3), (3, 5) . . . (5, 1), (5, 3), (5, 5). Inshort, all coefficients are encoded by four scans.

By generalizing this process as (dx=DX, dy=DY), the first scan in caseof part (1) in FIG. 12 encodes coefficients, as shown part (4) in FIG.12 at the positions of (0, 0), (0, DX), (0, 2*DX) . . . ;

the second scan encodes coefficients at the positions (0, 1), (0, DX+1),(0, 2*DX+1) . . . ;

the third scan encodes coefficients at the positions (0, 2), (0, DX+2) .. . (0, 2*DX+2) . . . ;

. . .

DX round scan encodes coefficients at the positions (0, DX−1), (0,2*DX−1), (0, 3*DX−1) . . . ;

(DX+1) round scan encodes coefficients at the positions (1, 0) (1, DX),(1, 2*DX) . . . ;

(DX+2) round scan encodes coefficients at the positions (1, 1) (1,DX+1), (1, 2*DX+1) . . . ;

(DX*DY) round scan encodes coefficients at the positions (DY−1, DX−1)(DY−1, 2*DX−1), (DY−1, 3*DX−1) . . . (2*DY−1, DX−1) (2*DY−1, 2*DX−1),(2*DY−1, 3*DX−1). In short, all coefficients are encoded by (DX*DY)scans. Part (2) in FIG. 12 is reverse to part (1) in FIG. 12 as tovertical and horizontal directions. The coefficients can be processed byscanning as shown in part (5) FIG. 12 at horizontal and verticalintervals of (dx=DX, dy=DY).

In comparison with coefficients encoded by raster scanning from the topleft of a frame, the subband coefficients encoded by scanning atintervals provided between the respective coefficients can reproduce animage whose content can be recognized at an earlier stage of decodingand which image can give better subjective impression by the effect ofgradually improving the quality of the decoded image.

An example of an integrated component unit containing a plurality ofsubbands included in the respective components is described below. FIG.13 shows another transmitting order of coefficients of the subband imageof FIG. 9. In a part of (b) of FIG. 13, there is presented an example ofencoding a subband image in four layers. Blocks (Y₂, Y₃, Y₄), (U₂, U₃,U₄) and (V₂, V₃, V₄) of in the part (b) of FIG. 13 consist each of three(m=3) subbands as shown in a part (a) of FIG. 13.

Namely, three subbands (1, 2, 3) shown in the part (a) of FIG. 13correspond to one element set in an integrated component unit. As thesethree subbands are treated as one element, three coefficients havingthe, same relative positions in respective three subbands are treated asone set as shown in a part (c) of FIG. 13.

Three coefficients existing at the same relative positions in therespective three subbands in FIG. 13 are supposed as one coefficient,i.e., a (coefficient Y_(HL1), coefficient Y_(LH1), coefficient Y_(HH1))are represented by a (coefficient Y₄). The component Y in each of thelayers shown in the part (b) of FIG. 13 is transferred first in thescanning order shown in FIG. 12. Similarly, the component U istransferred next and the component V is then transferred. Coefficientsof subband images of the same resolution levels in horizontal, verticaland diagonal directions are transmitted together from the coding side,so the resolutions of a reproduced image in horizontal, vertical anddiagonal directions are increased at a time at the decoding side.

According to another method for treating three subbands as one element,a subband 1 is first transmitted completely, a subband 2 is thentransmitted completely and a subband 3 is finally transmitted. Referringto FIG. 13, the subband 1 of the component Y is transmitted in thescanning order shown in FIG. 12. Next, the subband 2 of the component Yis transmitted in the same scanning order and then the subband 3 of thecomponent Y is transmitted in the same scanning order. Subsequently, thesubbands 1, 2 and 3 of the component U are transmitted one by one in thesame scanning order as that for the component Y. Finally, the subbands1, 2 and 3 of the components V are transmitted one by one in the samescanning order as that for the component Y.

An integrated component unit includes elements (the subband 1 of thecomponent Y, the subband 2 of the component Y, the subband 3 of thecomponent Y, the subband 1 of the component U, the subband 2 of thecomponent U, the subband 3 of the component U, the subband 1 of thecomponent V, the subband 2 of the component V, the subband 3 of thecomponent V). In this case, resolution of an image reproduced at adecoding side is increased in a horizontal direction, vertical directionand diagonal direction in the described order. Transmission of thesethree subbands in the order of subband 2, subband 1 and subband 3 causesan increase in resolution of the image in the horizontal, vertical anddiagonal directions in the described order at the decoding side.

Several examples of transmitting subband coefficients may be selectablyused. In case if a codable image is known to be of higher resolution ina specified direction, coefficients of a subband in the knownhigh-resolution direction are transferred first at the coding side andthe transferring order is rearranged at the decoding side to earlierreproduce the coefficients of the subband image in the knownhigh-resolution direction. This makes it possible to increase thequality of images in the decoding process at the decoding side. In thisinstance, it is necessary to inform the decoding side of thetransmitting order in which coefficients are encoded by placing suchinformation in the coded data.

The transmitting-order deciding/rearranging portion 104 shown in FIG. 7decides integrated component units (in the case of FIG. 9) to be ofsubbands (Y_(i), U_(i), V_(i)), where i=1 to 10 indicating the order oftransmitting subbands), rearranges subbands in the order of (Y₁,U₁,V₁),(Y₂,U₂,V₂), . . . , (Y₁₀,U₁₀,V₁₀) as shown in FIGS. 10 and 11 or in theorder of (Y₁,U₁,V₁), (Y₂,U₂,V₂), . . . , (Y₄,U₄,V₄) as shown in the part(b) of FIG. 13 and, then, outputs the rearranged coefficient-coded datato the variable-length coding portion 104.

Although the above-mentioned embodiment treats all coefficients in asubband as one group, it may also prepare an integrated component unitby using a coefficient or a plurality of coefficients in a subband as agroup. The following example treats one line in a subband as a group ofcoefficients.

FIG. 14 shows an integrated component unit consisting of horizontallines one in each of the subbands, which is expressed as follows:

(Y_(i)(y),U_(i)(y),V_(i)(Y)), where Y_(i)(y), U_(i)(y), V_(i)(y) areone-line data of respective subbands Y, U and V, i=1 to 10 indicates theorder of transferring subbands and y denotes each line number insubbands.

In this instance, the order of transferring the integrated componentunits is as follows: (Y₁(0), U₁(0), V₁(0)), (Y₁(1), U₁(1), V₁(1)), . . ., (Y₂(0), U₂(0), V₂(0)), (Y₂(1), U₂(1), V₂(1)), . . . , . . . ,(Y₁₀(10), U₁₀(0), V₁₀(0)), (Y₁₀(1), U₁₀(1), V₁₀(1)), . . . .

In FIG. 14, an integrated component unit consists of horizontal linesone in each of the subbands, which corresponds to the scanning ordershown in the part (a) of part (1) of FIG. 12. Besides this, it is alsopossible to prepare an integrated component unit composed of verticallines one in each of the subbands as shown in the part (a) of part (2)of FIG. 12. In this instance, components Y, U, V may be expressed eachby one arrow. This processing is done on all subbands in the order shownin FIG. 10, FIG. 11 or the part (b) of FIG. 13.

It is also possible to prepare an integrated component unit composed ofcoefficients one in each of the subbands, which is expressed as:

-   (Y_(i)(y,x), U_(i)(y,x), (V_(i)(y,x)), where Y_(i)(x,y), U_(i)(x,y),    V_(i)(x,y) are coefficients one in respective subbands Y, U and V,    i=1 to 10 indicates the order of transferring subbands, y denotes a    position in vertical direction in a subband, x denotes a position in    horizontal direction in a subband. In this instance, the    transmitting order may be any of the orders shown in parts (1), (2)    and (3) of FIG. 12. The processing is made on all subbands in the    order shown in FIGS. 10, FIG. 11 or the part (b) of FIG. 13.

FIG. 15 is a flow chart depicting an example of the operation of thetransmitting order deciding and rearranging portion 104 of FIG. 7. Inthe shown case, the integrated component unit may be changed over fromthe subbands to coefficients (one or more groups of coefficient) or viceversa. The portion may be designed to operate by using only one of thetwo units.

The video coding device according to the first embodiment of the presentinvention can prepare coded data having a hierarchical structure bydecomposing an input image composed of a plurality of components Y, Uand V into subband images and encoding the subband images in anascending order of their resolution starting from the lowest-resolutionsubband.

In comparison with the conventional device that integrates components Y,U and V according to only the scanning line base, the first embodimentof the present invention can perform adaptive encoding input video datain view of the data characteristics by applying integrated componentunits according to subband-based and/or coefficient-based integration ofthe components Y, U and V.

Referring now to FIG. 8, a video decoding device embodying the presentinvention will be described below in detail. This video decoding deviceis intended to decode video data prepared by the video coding deviceaccording to the first embodiment of the present invention.

In FIG. 8, the video-decoding device comprises a variable-lengthdecoding portion (variable-length decoding means) 201, a decoded-datacounting portion (decoded data counting means) 202, a decodingtruncating portion (decoding truncating means) 203, a data completingportion (data completing means) 205, a coefficient decoding portion(coefficient decoding means) 206 and a subband synthesizing portion(subband synthesizing means) 207. These portions are similar inconstruction to the portions a variable-length decoding portion 2101, adecoded data counting portion 2102, a decoding truncating portion 2103,a data completing portion 2105, a coefficient decoding portion 2106 anda subband synthesizing portion 2107, respectively, of FIG. 4.

In FIG. 8, numeral 204 designates a component separating portion(component separating means, arranging means) for separatingcoefficient-coded data rearranged by the transfer-order deciding andrearranging portion of the coding device into data for respectivecomponents. The component separating portion 204 rearranges coded datainto respective component groups Y, U and V by inverting the procedurethat the coding side did.

Accordingly, the component separating portion 204 has a memory (notshown) for storing respective components Y, U and V. This memory hasbasically the same capacity that the subband decomposing portion 101 ofthe video coding device has. However, this portion may be designed toseparate an integrated component unit into coefficients for respectivecomponent groups Y, U, V and output the separated coefficients to thecoefficient decoding portion 206 on completion of separation of theintegrated component unit. In this instance, the portion may have thememory enough to store the largest integrated-component unit only.

The component separating portion 204 may also be designed to work bysuccessively separating and outputting coefficients of an integratedcomponent unit to the coefficient decoding portion 206. In this case,the component separating portion 204 may have a memory enough to store aplurality of separated coefficients irrespective of the size of anyintegrated component unit to be separated. The separated coefficientsoutputted from the component separating portion 204 are decoded by thecoefficient decoding portion 206 and then stored in a memory (not shown)of the coefficient decoding portion 206. In this case, the coefficientdecoding portion 206 includes decoded-data arranging means.

Furthermore, it is also possible to write the decoded coefficients in amemory (not shown) for storing frequency-coefficients to be input intothe subband synthesizing portion 207.

As described above, the coefficients separated by the componentseparating portion 204 may be stored in a variety of memory means. Forthe convenience of further explanation, the component separating portion204 separates integrated component units by writing separatedcoefficients in its memory having the same capacity as the memory of thesubband decomposing portion 204 of the video coding device has.

For example, an integrated component unit composed of subband data(Y₁,U₁,V₁) is decomposed into separate elements Y₁, U₁ and V₁respectively. The separated elements Y, U and V are written intocorresponding subbands positions in a memory. Next, a unit (Y₂, Y₂, V₂)is decomposed into separate elements Y₂, U₂ and V₂ that are then writtenin specified positions of the corresponding subbands in the memory forstoring Y, U and V. This processing is done on all the subbands.

In this instance, the order of decoding coefficients in each subband isthe same as described the process shown in FIG. 12. For example, thecoding side performed scans as shown in the part (1) of FIG. 12, so thedecoding side must do scans as shown in the part (1) of FIG. 12. Theapplication of this scanning method causes an image in the decodingprocess to have resolution increasing in the order of raster scanningfrom the top left to the down right.

In a particular case when coefficients were encoded by scanning withspacing between them as shown in the part (4) or (5) of FIG. 12, animage being decoded can be easily recognized at an earlier stage ofdecoding as compared with the raster scanned image. The image may begradually improved in resolution level, so the image may have bettersubjective-image quality.

The operation of the component separating portion 204 when processing anintegrated component unit composed of one or more groups (sets) ofcoefficients in subbands is as follows:

Assuming one line in a subband is considered as a group of coefficients,an integrated component unit is expressed as (Y_(i)(y), U_(i)(y),V_(i)(y)) where Y_(i)(y), U_(i)(y), V_(i)(y) are one-line data ofrespective subbands Y, U and V, i=1-10 (the order of transmittingsubbands) and y designates a position in a vertical direction in asubband.

Y_(i)(y), U_(i)(y), V_(i)(y) are separated from each other and writtenin positions <line y> of the corresponding subbands i in the memory forstoring Y, U and V. This processing is done on all the lines in thesubbands in the order from the lowest-resolution subband to thehighest-resolution subband.

When an integrated component unit is composed of coefficients selectedone from each subband, it is expressed as: (Y_(i)(y, x), U_(i)(y, x),V_(i)(y, x)) where Y_(i)(y, x), U_(i)(y, x), V_(i)(y, x) aresingle-coefficient data of respective subbands Y, U and V, i=1-10 (theorder of transmitting subbands) and y designates a position in avertical direction in a subband and x designates a position in ahorizontal direction in a subband.

Elements Y_(i)(y, x), U_(i)(y, x), V_(i)(y, x) are separated from eachother and written in positions (y, x) of a coefficient of thecorresponding subbands i in the memory for storing Y, U and V. Thisprocessing is done on all the coefficients in the subbands from thelowest-resolution subband to the highest-resolution subband.

As described before referring to FIG. 12, the decoding side applies thesame coefficient-scanning method as the coding side used even if anintegrated component unit is selected by 1 line or by one coefficient.

The separate coefficients outputted from the component separatingportion 204 are combined into respective component groups Y, U and V andthen treated as respective groups.

FIG. 16 is a flow chart depicting an exemplified operation procedure ofthe component separating portion 204. In the instance shown in FIG. 16,the integrated component unit may be changed over from the subbands tocoefficients (one or more groups of coefficient) or vice versa. Theportion 204 may be provided with either one of the two integratedcomponent units.

Referring to FIG. 8, the operation of the data completing portion 205will be described below in detail:

In this case, an integrated component unit is composed of one-line datain a subband.

Now let us suppose that the decoding operation was stopped by the actionof the decoding truncating portion 203 because the number of bits of thedecoded data exceeded a threshold value when data for instance one-linedata of integrated component unit shown in FIG. 14, e.g., (Y₁(0), U₁(0),V₁(0)), . . . , (Y₃(5), U₃(5), V₃(5)) has been decoded. In thisinstance, coefficients of the subbands 1 and 2 and coefficients of thefirst line to the fifth line of the subband 3 have been decoded but theremaining parts have no coefficient.

Data completing portion 205 produces subband coefficients by putting 0in remaining vacant parts where no data exist. This enables thecoefficient decoding portion 206 and the subband synthesizing portion207 to normally perform subsequent processing steps. Vacant data mayalso be replaced with any other value than 0. The provision of thedecoded data counting portion 202, the decoding truncating portion 203and the data completing portion 205 enables the decoding side totruncate the decoding operation at any position by user's request.

The data completing portion 205 do nothing while the number of bits ofthe decoded data does not exceed the threshold value. Accordingly, amaximally expressible value may be previously set as the threshold valuein case of decoding all the coded data. The embodiment may be a systemof FIG. 8, which in this instance omits the decoded data countingportion 202, the decoding truncating portion 203, the data completingportion 205.

The coded data of the hierarchical structure, which represents an imagecomposed of a plurality of components Y, U, V, can be decoded at thedecoding side. The coded data of the hierarchical structure allows theprogressive decoding the coded data, whereby the quality of the entirereproduced image is sequentially improved. As compared with the priorart method limited to the integrated component units of lines one foreach component, the present invention enables the system to conduct avariety of progressive reproduction of the coded image. For example, thepresent invention method can uniformly improve the resolution of thereproduced image in horizontal, vertical and diagonal directions,whereas the prior art method improves the image resolution in thehorizontal direction before the other directions.

A second embodiment of the present invention for coding and decoding animage composed of components having different sizes is described below.The second embodiment of the present invention is similar inconstruction to the first embodiment except for the operation of thetransfer-order deciding and rearranging portion 104 (step of outputtingan integrated component unit). Therefore, the same portions as the firstembodiment are not explained further. Only the transfer-order decidingand rearranging portion 104 will be described below.

For example, a format of 4:2:0 in which components U and V have ahorizontal and vertical size being one-half that of the component Y isused. FIG. 17 shows resultants of three times of band-decomposition ofeach the component Y whose size is Y*X and the components U and V whoserespective size is Y/2*X/2. In this case, the subbands U and V are eachhalf the size of the component Y in horizontal and vertical directions.Respective components Y, U and V have the same number of subbands.

Accordingly, no problem arises with subband-based integrated units whenapplying the same scanning method and the same transmitting order as thefirst embodiment used. However, there may be a trouble with anintegrated component unit composed of one or a plurality of coefficient.

The subbands Y, U, V are different in size each other, so thelarge-sized component Y may have excess coefficients if an integratedcomponent unit is composed of the same number of coefficients persubband as described before in the first embodiment of the presentinvention. Therefore, the number of coefficients per component to beincluded in an integrated component unit is set according to the sizeratio of respective components if the components are different from eachother in size.

A part (1) of FIG. 18 shows the corresponding of coefficients ofrespective components for an image in the format of 4:2:0 as shown inFIG. 17. As the components Y, U and V have the same number of subbandsper component, the order of transferring the integrated component unitsis the same as shown in FIG. 10 for the first embodiment of the presentinvention. In this instance, the ratio of the horizontal and verticallengths of the components Y, U and V is of 2:1:1, so one coefficient ofeach component U or V corresponds to 4 coefficients of the component Y.Accordingly, the numbers of coefficients for the components Y, U, V tobe included in an integrated component unit is determined according tothe ratio of 4:1:1.

The integrated component units prepared on the basis of the subbands ofrespective components contain coefficients of the components Y, U and Vat the ratio of 4:1:1 and are processed in the same manner as describedbefore for the first embodiment of the present invention.

Accordingly, an integrated component unit for one line per subband isprepared to contain two lines Y, one line U and one line V as shown inFIG. 19 while an integrated component unit for a coefficient group persubband is prepared to contain 2*2 coefficients Y, one coefficient U andone coefficient V as shown in FIG. 20. In FIG. 19, numerals suffixed oneto component data Y, U and V indicate, by way of example, the order oftransferring the data within the respective integrated component units.The integrated component units for Y, U and V are processed one by onefor one subband. On completion of processing one subband, the processadvances to processing another subband in the order shown in the part(1) of FIG. 18. The order of transferring the subbands may be either oneof those shown in FIGS. 11 and the part (b) of FIG. 13.

With differently sized components Y, U and V, the video decoding sidecan decode the coded data transmitted from the coding side by changingthe numbers of data of components contained in an integrated componentunit according to the ratio of horizontal and vertical sizes of thecomponents and writing the data in the corresponding positions in amemory.

The same scanning method and the same transferring order as describedfor the first embodiment can be applied in this embodiment when workingwith the subband-based integrated component units.

The second embodiment of the present invention is similar inconstruction and function to the first embodiment except for theoperation of the transfer-order deciding and rearranging portion 204(step of separating an integrated component unit into respectivecomponents and writing separated data in corresponding subbands in amemory). Therefore, the further description is omitted.

In the second embodiment of the present invention, it is possible togive coded data a hierarchical structure even if components of an imagehave different sizes. The decoding side can decode entire decoded dataand can also obtain an entire reproduced image from a part of the codeddata.

Although the second embodiment has been described with only an imagehaving components whose horizontal and vertical size ratio is 2:1:1, itcan treat other size ratios of image components in the similar manner asdescribed above.

A third embodiment of the present invention is adaptable to the case ofprocessing image components being different in size and decomposed intodifferent numbers of decomposition levels by the subband decomposingportion 101 of FIG. 7. This embodiment of the present invention issimilar to the first embodiment except for the operation of thetransfer-order deciding and rearranging portion 104 (step of outputtingan integrated component unit). Therefore, the same portions are notexplained further. Only the transfer-order deciding and rearrangingportion 104 will be described below.

Referring to FIG. 21, this embodiment is described by way of examplewith an input image whose components U and V have horizontal andvertical lengths being one half of those of the component Y. A partdesignated by (1) of FIG. 21 shows the results of decomposing thecomponent Y three times and the components U and V twice respectively.As the number of subbands of Y differs from that of U and V, the thirdembodiment cannot use the transferring methods described for the firstand second embodiments and so uses the following method of transferringthe subbands.

An integrated component unit composed of subbands of respectivecomponents Y, U and V is first described. In this instance (firstexample), combinations of three-component subbands Y, U and V, preparedfrom sevens of low-resolution subbands of respective components Y, U andV as shown in a part (2) of FIG. 21, are transferred one by one to thevariable-length coding portion 103 of FIG. 7, then three remainingsubbands of the component Y are transferred independently one by one tothe variable-length coding portion 103. In short, each one oflow-resolution subbands of respective components Y, U and V, zero piecesof high-resolution subbands of the components U and V and only one ofhigh-resolution subband of the component Y compose respective integratedcomponent units to be output.

Accordingly, the transferring order is expressed as (Y₁,U₁,V₁),(Y₂,U₂,V₂), . . . , (Y₇,U₇,V₇), (Y₈), (Y₉), (Y₁₀).

In this instance, the subbands of respective Y, U, V component (Y_(i),U_(i),V_(i)), where i=1 to 7 designates the transferring order, have thesame sizes, so the same scanning method as used in the first embodimentcan be applied to the case that integrated component unit is subbands orone or more groups of coefficient.

Subbands of components Y, U and V are combined by threes as shown in thepart (b) of FIG. 13 to form combinations:

-   (Y₁,U₁,V₁), (Y₂,Y₃,Y₄, U₂,U₃,U₄, V₂,V₃,V₄), (Y₅, Y₆,Y₇, U₅,U₆,U₇,    V₅,V₆,V₇), (Y₈), (Y₉), (Y₁₀).

Another example (second example) is shown in FIG. 22. The subband imagesshown in a part (1) of FIG. 22 are obtained as the result of decomposingthe component Y three times and the components U and V twicerespectively. In this instance, integrated component units to betransmitted are formed by combining a set of four low-resolutionsubbands Y with two low-resolution subbands U and V as shown in a part(2) of FIG. 22 and by combining high-resolution subbands of thecomponents Y, U and V with each other by ones as shown in a part (3) ofFIG. 22.

The order of transferring the subbands is as follows: (Y₁, Y₂, Y₃, Y₄,U₁, V₁), (Y₅, U₂, V₂), (Y₆, U₃, V₃) (Y₇, U₄, V₄), (Y₈, U₅, V₅), (Y₉, U₆,V₆), (Y₁₀, U₇, V₇).

In the second example, three components have corresponding subbandshaving different sizes, so the same scanning method as used in thesecond embodiment can be applied to each integrated component unit.

Subbands (Y₅, Y₆, Y₇) shown in the part (3) of FIG. 22 corresponds to avertical-resolution subband, a horizontal-resolution subband and adiagonal-resolution subband respectively and have the same resolutionlevels. Therefore, each integrated component unit can be prepared bycombining respective high-resolution subbands of respective componentsY, U and V by threes for each component. This is the third example ofpreparing an integrated component unit. Similarly, an integratedcomponent unit may be prepared of combinations (Y₈, Y₉, Y₁₀), (U₂, U₃,U₄), (U₅, U₆, U₇), (V₂, V₃, V₄), (V₅, V₆, V₇).

In this instance, the transmitting order is as follows: (Y₁, Y₂, Y₃, Y₄,U₁, V₁), (Y₅, Y₆, Y₇, U₂, U₃, U₄, V₂, V₃, V₄), (Y₈, Y₉, Y₁₀, U₅, U₆, U₇,V₅, V₆, V₇).

In the above-described example, subbands Y₁, Y₂, Y₃, Y₄, U₁, V₁ areselected as a plurality of low-resolution subbands with keeping the sizeratio of Y, U and V (in this example, the horizontal and vertical sizeratio is 2:1:1). Besides the above combination, Y₁, Y₂, . . . , Y₇, U₁,. . . , U₄, V₁ . . . , V₄ may also be selected as a plurality of thelow-resolution subbands.

A further fourth example which is another variety of above-mentionedsecond or third example is such that the lowest resolution subbands Y,U, V is extracted from respective groups of low-resolution subbands Y₁,Y₂, . . . , Y₇, U₁, . . . , U₄, V₁ . . . , V₄ for respective componentsand separately transferred as shown in a part (4) of FIG. 22 and a setof remaining low-resolution subbands is then transferred like second orthird example. In this instance, the set of the low-resolution subbandsis a combination of six Y-component subbands, three U-component subbandsand three V-component subbands as shown in a part (5) of FIG. 22. Aftertransmission of all the low-resolution subbands, high-resolutionsubbands shown in a part (6) of FIG. 22 are transferred.

In the case of combining subbands one for each component (correspondingto the second example), the order of transferring the subbands is asfollows: (Y₁, U₁, V₁), (Y₂, Y₃, Y₄, Y₅, U₂, V₂), (Y₆, U₃, V₃), (Y₇, U₄,V₄), (Y₈, U₅, V₅), (Y₉, U₆, V₆), (Y₁₀, U₇, V₇).

In the case of combining subbands by threes for each component(corresponding to the third example), the order of transferring thesubbands is as follows: (Y₁, U₁, V₁), (Y₂, Y₃, Y₄, Y₅, Y₆, Y₇, U₂, U₃,U₄, V₂, V₃, V₄), (Y₈, Y₉, Y₁₀, U₅, U₆, U₇, V₅, V₆, V₇).

For components having subband decomposition levels of not less than 4,subbands having resolution levels higher than that of a subband shown ina part (6) of FIG. 22 are the same in quantity for components Y, U andV. In this instance, integrated component units composed each of acombination of the same number of subbands Y, U, V are subsequentlytransmitted.

In the above examples from first to fourth, the transmitting order ofthe three subbands which have the same resolution levels in horizontal,vertical and diagonal directions shall not be restricted. In otherwords, for three subbands which have the same resolution levels as shownin a part (a) of FIG. 13, the transmitting order of these three subbandsmay be not only subband1, subband2, subband3, but also subband2,subband1, subband3, for example.

In the first example, the video decoding side can decode the coded datareceived from the coding side by separating components in eachintegrated component unit into groups of respective components Y, U andV, writing separated data in the corresponding subband areas in thememory and finally writing three highest-resolution subbands in thecorresponding subband area of the memory.

In the second example, the video decoding side can decode the coded datareceived from the coding side by separating the subbands in eachintegrated component unit by reversing the process made by the codingside and writing the subbands of each component in the correspondingsubband area in the corresponding memory. In this case, the thirdembodiment differs from the first and second embodiments by the factthat only the first integrated component unit is prepared from aplurality of the low-resolution subbands by the coding side and containsfour subbands Y, one subband U and one subband V.

As described above, the number of low-resolution subbands to be combinedwith each other can be freely selected. For example, an initialintegrated-component unit may contain seven Y-component subbands, fourU-component subbands and four V-component subbands.

In the third example, each integrated component unit can be decomposedinto subbands of respective components Y, U and V by the same method asdescribed for the second example and written into corresponding subbandareas (for Y, U and V components) in the memory for decoding them. Sincethe high-resolution subbands Y, U and. V have been combined by threes inan integrated component unit at the coding side, the high-resolutionsubbands Y, U and V are recorded by threes for each component incorresponding subband areas in the memory for decoding them. Thisexample differs from the second example by the above-mentioned feature.

In the fourth example, the video decoding side can decode the coded datareceived from the coding side by separating the subbands in eachintegrated component unit in the same manner as shown for decodingdevice in second or third example and writing the subbands of eachcomponent in the corresponding subband area in the corresponding memory.This example differs from the second and third examples by the fact thatthe lowest-resolution subbands of the respective components Y, U, V arefirst stored in the corresponding subband areas (Y, U, V) in the memoryfor decoding and then low-resolution subbands other than thelowest-resolution subbands are stored in the corresponding areas (Y, U,V) in the memory.

The video decoding device according to the third embodiment of thepresent invention is similar to that of the first embodiment except forthe operation of the component separating portion 204 (step ofseparating each integrated component unit into respective components andwriting the separated data in the corresponding areas in a memory).Therefore, further explanation is omitted.

With integrated component units each composed of one or more groups ofcoefficients such as lines instead of the subbands for respectivecomponents Y, U and V, the third embodiment can apply the same processas described before with the same case in the first and secondembodiments based upon the same method as described for the integratedcomponent unit composed the subbands for the respective components Y, Uand V.

In the third embodiment of the present invention, an image whosecomponents have different sizes and different decomposition levels canbe encoded so that coded data having a hierarchical structure isobtained at the coding side and an entire image is reproduced from theentire coded data or a part of the coded data at the decoding side.

Although the third embodiment has been described by way of example withonly an image having components whose size ratio is of 2:1:1, it cantreat other size ratios of image components in the similar manner asdescribed above. For example, an image whose components Y, U and V arethe same in size and have different numbers of subbands can be encodedto have a hierarchical structure through the same process as describedabove in the third embodiment. The transmitting orders corresponding tothose shown in FIGS. 11 and 13 are also adopted besides the describedorder of FIG. 10.

The three embodiments of the present invention have been described byway of example with the specified order of transferring the elements Y,U, V in the integrated component units but shall not be limited to thatorder.

INDUSTRIAL APPLICABILITY

The present invention brings following advantageous effects.

Firstly, the video coding device and the video decoding device accordingto the present invention operate with integrated component units whoseelements are all frequency coefficients in m (m>1) respective subbandsof respective component A^(n) and can therefore transmit and decodedfirst specified subbands of the image components that may be componentsY, U and V and have different levels of influence on human visualproperty, allowing one to recognize an essence of the image at anearlier stage of decoding at the decoding side. When a codable image isknown to be of higher resolution in a specified direction, the codingdevice can transmit first coded coefficients ofhigher-resolution-direction subbands and the decoding device can decodethose coded coefficients, and even in case of terminating the decodingin the midway of decoding all coded data, can thereby reproduce theimage from only data decoded till that time to improve subjective-imagequality of the image.

Secondly, the video coding device and the video decoding deviceaccording to the present invention operate with integrated componentunits whose elements are m (m>1) pieces of frequency coefficients havingthe same relative positions in m (m≧1) respective subbands of respectivecomponents A^(n) and can decode those coded coefficients, and even incase of terminating the decoding in the midway of decoding all the codeddata, can thereby reproduce the image from only the data decoded tillthat time to improve subjective-image quality of the image when theimage is composed of components R, G and B that have substantiallyalmost the same influence on human visual property.

Thirdly, the video coding device and the video decoding device accordingto the present invention can be adapted to process an image whoseluminance and chrominance components are different from each other byresolution, having a great advantage over the conventional method thatcan be applied to an image whose components have the same resolution.This feature provided by the present invention is very desirable inparticular to digital image processing since many digital images areusually formatted to have higher resolution of the luminance componentthan that of chrominance component.

Fourthly, the video coding device and the video decoding deviceaccording to the present invention can be adapted to process an imagewhose luminance and chrominance components are different from each otherby resolution and have different subband-decomposition levels, getting agreat advantage over the conventional method that can be applied to animage whose components have the same resolution and the same number ofsubbands. This feature provided by the present invention is verydesirable in particular to digital image processing since many digitalimages are usually formatted to have higher resolution of the luminancecomponent than that of chrominance component and it is general to usethe different subband-decomposition levels according to the components'resolution levels.

Fifthly, the video coding device and the video decoding device accordingto the present invention can be adapted to process an image whoseluminance and chrominance components have different resolution levelsand different subband-decomposition levels, getting a great advantageover the prior art method that can be applied to an image whosecomponents have the same resolution and the same number of subbands.Furthermore, these aspects of the present invention provide such afeature that each integrated component unit always reflects the ratio ofthe number of respective components contained in an input image. Thisfeature eliminates the need for decoding, redundant data at the decodingside when decoding the amount of data according to the resolution of thedisplay unit.

Sixthly, in addition to the fifth advantageous effect above-mentioned,the video coding device and the video decoding device according to thepresent invention can first separate and transmit lowest-resolutionsubbands of respective components A^(n) to first present the summary ofan image, making it possible to improve subjective quality of thereproduced image.

1-12. (Canceled).
 13. A video coding device comprising: asubband-decomposing means for decomposing into subbands each of aplurality of components composing an image; a coefficient coding meansfor encoding frequency coefficients in the subbands to generate codedcoefficient data for each subband; and a variable-length coding meansfor performing variable-length coding of the coded coefficient data,wherein the number of subbands of at least one component is differentfrom the number of subbands of other components.
 14. A video codingdevice comprising: a subband-decomposing means for decomposing intosubbands each of a plurality of components composing an image; acoefficient coding means for encoding frequency coefficients in thesubbands to generate coded coefficient data for each subband; and avariable-length coding means for performing variable-length coding ofthe coded coefficient data, wherein an image size and the number ofsubbands of at least one component are different from an image size andthe number of subbands of other components.
 15. A video coding device asdefined in any of claims 13 and 14, wherein the sets of codedcoefficient data of the subbands for each of the components aretransferred in an ascending order of frequencies starting from a lowfrequency subband.
 16. A video decoding device comprising: avariable-length decoding means for performing variable-length decodingof coded data of an image composed of a plurality of components toobtain coded coefficient data; a coefficient decoding means for decodingthe coded coefficient data to obtain frequency coefficients; and asubband synthesizing means for reconstructing a decoded image bysynthesizing subbands of the frequency coefficients for respectivecomponents, wherein the number of subbands of at least one component isdifferent from the number of subbands of other components.
 17. A videodecoding device comprising: a variable-length decoding means forperforming variable-length decoding of coded data of an image composedof a plurality of components to obtain coded coefficient data; acoefficient decoding means for decoding the coded coefficient data toobtain frequency coefficients; and a subband synthesizing means forreconstructing a decoded image by synthesizing subbands of the frequencycoefficients for respective components, wherein an image size and thenumber of subbands of at least one component are different from an imagesize and the number of subbands for other components.
 18. A videodecoding device as defined in any of claims 16 and 17, wherein the setsof coded coefficient data of the subbands for each of the components arereceived in an ascending order of frequencies starting from a lowfrequency subband.
 19. An image decoding device comprising: avariable-length decoder for decoding variable-length coded data of ahierarchically coded image, said hierarchically coded image includingmore than one kind of luminance or chrominance components; a decodingtruncator for giving a decoding-stop command at any position designatedby a user; a coefficient decoder for decoding the variable-length codeddata; and a subband synthesizer for reconstructing a decoded image bysynthesizing subbands containing data decoded by the coefficient decoderfor said components, wherein an image decoded with truncated data isdisplayable by using data decoded before terminating decoding of saidimage.
 20. An image decoding device comprising: a variable-lengthdecoder for decoding variable-length coded data of a hierarchicallycoded image, said hierarchically coded image including more than onekind of luminance or chrominance components; a decoding truncator forgiving a decoding-stop command at any position designated by a user; acoefficient decoder for decoding the variable-length coded data; and asubband synthesizer for reconstructing a decoded image by synthesizingsubbands containing data decoded by the coefficient decoder for saidcomponents, wherein an image decoded with truncated data is displayableby using data decoded before terminating decoding of said image when atleast one kind of said components is different in size and in number ofsubbands from other kinds of said components.